Estimating the Entropy Rate of Spike Trains

Information-theoretic methods have been widely used in neuroscience, in the broad effort to analyze and understand the fundamental informationprocessing tasks performed by the brain. In these studies, the entropy has been adopted as the main measure for quantifying the amount of information transmitted between neurons, via the spike trains they generate. One of the first and most important goals is to identify appropriate methods that can be used to quantify the amount of information that gets communicated by spike trains, or, in other words, to estimate the entropy of spike trains recorded from live animals. So far, the most commonly used entropyestimation technique has been the so-called “plugin” (or maximum-likelihood) estimator and its various modifications. This method consists of essentially calculating the empirical frequencies of all words of a fixed length in the data, and then estimating the “true” entropy of the underlying signal as the entropy of this empirical distribution; see, e.g., [10][5][12][6][9]. For computational reasons, the plug-in estimator cannot go beyond word lengths of about 10 or 20, and hence it does not take into account the potential longer time dependencies in the signal. Here we examine the performance of entropy estimators based on two data compression algorithms, the Lempel-Ziv algorithm (LZ) and the Context Tree Weighting method (CTW). Specifically, we consider two LZ-based entropy estimators and one based on the CTW. The first LZ-based method has been widely and very successfully used in many applications, and the other one is a new estimator with some novel and more desirable statistical properties. The CTW-based estimator is based in the work of Willems et al [13][14][15] and it has also been considered in [1][3].